It is a usual practice for companies providing access to the Internet and for companies providing content and services on the Internet to generate logs of access and activity. Some examples of how logs are used are: for debugging and troubleshooting, detection and monitoring of abuse, statistical analysis, demographic analysis, report generation and other general business purposes.
One issue that arises with the storage of access and activity logs is the convenient and efficient maintenance of those logs. Entities that provide access to the Internet, and entities that make content and services available on the Internet, often have the triple responsibilities of (1) maintaining privacy, (2) maintaining the integrity of the services being provided, and (3) complying with all applicable laws regarding the disclosure of information. To fulfill the responsibility of maintaining privacy, the entity would ideally log as little information as possible. Any information maintained represents a liability to the entity generating the information in this regard since it represents a risk of disclosure and possible compromise of privacy.
To fulfill the second responsibility of maintaining the integrity of the services being provided, the entity needs to log certain types of information for certain periods of time. For example, enough information should be maintained long enough so that abuse can reasonably be detected over a reasonable period of time. Additionally, billing requirements may require certain information be maintained. This responsibility does not necessarily mean complete logs must be maintained. In certain cases, the entity only needs summary information, and/or only needs to maintain the information for a limited period of time. For example, an entity proving access to the Internet may want to maintain for over a year the number of minutes connected on a certain day, while the specific IP address in use on that day and the specific port numbers used may only be needed for days or weeks.
The third responsibility is to comply with all applicable laws for the jurisdiction under which the entity operates. In some cases, there are no laws that require the preservation of logging information, in which case the logged information would be governed by other concerns. In other cases regulations may require that certain types of information may be maintained for a specific period of time. Another type of legal responsibility that arises in certain circumstances is not a requirement a priori that certain information be maintained, but that all information that is under the custody or control of the entity is produced at the time a subpoena is received. Since responding to subpoenas is expensive and time consuming, it is most efficient to maintain custody and control only of that information that is required for business reasons or legal reasons.
Thus, it can be seen that there is a balance in what information is logged, how it is accessed and for how long it is maintained. In many cases merely maintaining complete logs indefinitely is not an efficient or appropriate mechanism for maintaining these multiple responsibilities. What is needed is an improved method of maintaining and accessing logs that allows log generating entities to more cost effectively balance their multiple responsibilities.